The present invention provides for a method of transforming over time (or temporally resampling) a set of input frames from a digital cinema camera to produce an output sequence of frames with different temporal sampling, meaning different playback frame rate, variable apparent motion rate, or a combination of both.
Digital cinema cameras are not often capable of “speed ramps,” which are commonly used with film cameras. In a film camera, the frame rate of acquisition is changed continuously during a shot, allowing the apparent speed of motion of actors or objects in the scene to vary during constant frame rate playback. The artistic uses of this technique were widely varied, but the limitations of digital cameras have prevented this technique.
Current methods of temporal resampling are limited by their ability to generate a proper transform function and to apply a finite impulse response (FIR) filter.
The transform function is the function that provides the mapping between the input frame set and the output frame set and vice versa. It is desired that the transform function be user variable, but it is also desirable that the transformation function map integer frame numbers as often as possible. These are sometimes contradictory requirements, as allowing the user full control of the resampling function, and allowing the resampling function to have a smooth derivative will usually result in non-integer mapping of frame numbers.
Once transformed from input to output frames, there is still the problem of properly resampling the input frames surrounding the time center to create an output frame with good temporal response. This involves the use of a finite impulse response (FIR) filter to perform a weighted average of input frames for each output frame. Proper FIR creation often results in weighting with negative coefficients. As a principle of signal processing, this is mathematically correct, but in practice the negative weighting coefficients present a problem when very bright highlights in the image set are later clipped in editing. The negative values created from these very positive input data can present unwelcome artifacts when the positive data is later clipped. Restricting FIR coefficients to only positive values dramatically limits the possible frequency responses available for resampling.
Current methods of frame rate resampling rely upon simple nearest-neighbor mapping of input to output frame numbers, and in a speed ramp often result in stuttered or jerky motion. The present invention optimizes the temporal relationship between the input and output frames, and by adjusting the weighting of the FIR filter, achieves a much more accurate motion look. The smooth FIR waveform, in turn, can result in negative image values, and the damped subtraction in the present invention alleviates potential discontinuities based on sensor clipping or color adjustment in post production.